Skip to content

Conversation

@dervoeti
Copy link
Member

@dervoeti dervoeti commented Oct 14, 2024

Description

Fix needed for stackabletech/issues#614

Changes:

  • Changed some RUN commands to Heredoc
  • Break circular dependencies in the Airflow SBOM by removing the dependencies from the providers on Airflow itself. For example, in the current SBOM, apache-airflow-providers-smtp depends on apache-airflow and apache-airflow depends on apache-airflow-providers-smtp. This causes the rendering of the dependency tree in SecObserve to fail.
  • Create an SBOM for Kafka. It took a while to figure out how to not include all the integration tests in the SBOM, this was the only working solution I found.
  • Generate build-time SBOMs for OPA and statsd_exporter. I had to create a dummy Git repo with a dummy commit that's tagged with the version of OPA / statsd_exporter, because that's how cyclonedx-gomod (and I think Go in general) determines the version at build time. I did not find another way, I checked the source code of cyclonedx-gomod, that seems to be the only valid way. Since we don't include the .git folder in our .tar.gz archives in Nexus, a dummy Git repo has to be created.
- [x] Changes are OpenShift compatible
- [x] All added packages (via microdnf or otherwise) have a comment on why they are added
- [x] Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
- [x] All packages should have (if available) signatures/hashes verified
- [ ] Add an entry to the CHANGELOG.md file
- [ ] Integration tests ran successfully

@dervoeti dervoeti requested a review from a team October 15, 2024 08:16
@siegfriedweber siegfriedweber requested review from siegfriedweber and removed request for a team October 15, 2024 12:51
@dervoeti
Copy link
Member Author

@siegfriedweber Thanks for the review so far, that was very helpful. I should have checked things more thoroughly, sorry, will do better next time.
I now also pinned the versions of s3fs and cyclonedx-bom to the current versions, they were not pinned before, which means the latest version gets installed (for example version 5.0.0 of cyclonedx-bom came out 19 hours ago and got installed right away).

@lfrancke
Copy link
Member

Sorry for the conflicts I caused.
When rebasing please make sure that there are no COPY or other instructions changing permissions after the chown/chmod combo. Feel free to ping me if you're unsure.

@dervoeti dervoeti force-pushed the feat/add-missing-sboms branch from 2fb698c to b4c5d1c Compare October 16, 2024 10:12
@dervoeti dervoeti force-pushed the feat/add-missing-sboms branch from b4c5d1c to c459c49 Compare October 16, 2024 10:15
@dervoeti
Copy link
Member Author

@lfrancke @siegfriedweber I rebased my changes on top of the current main branch now

Copy link
Member

@siegfriedweber siegfriedweber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dervoeti dervoeti added this pull request to the merge queue Oct 17, 2024
Merged via the queue into main with commit 094be62 Oct 17, 2024
2 checks passed
@dervoeti dervoeti deleted the feat/add-missing-sboms branch October 17, 2024 09:02
Techassi pushed a commit that referenced this pull request Oct 21, 2024
* feat: generate SBOMs at build time for OPA, statsd_exporter and kafka

* fix: remove circular dependencies in Airflow SBOM

* fix: kafka: ignore test components in SBOM

* fix: kafka: missing patchfile for kafka 3.8.0

* fix: no need to cleanup builder image

Co-authored-by: Siegfried Weber <[email protected]>

* fix: add comment about cyclonedx-gomod to statsd_exporter as well

Co-authored-by: Siegfried Weber <[email protected]>

* fix: undo merge errors

* fix: casing to make linter happy

* fix: indenting and alphabetical sorting of packages

Co-authored-by: Siegfried Weber <[email protected]>

* fix: re-added line to remove sourcecode after build of statsd_exporter

* fix: merge RUN layers in statsd_exporter

* fix: place SBOM files closer to the application they are for

* fix: update gradle cyclonedx plugin to version 1.10.0

* fix: remove unnecessary curl flags, because we have a curlrc file

* fix: fixed variable substitution

* feat: pinned versions of python packages

* fix: fixes to adapt upstream changes

* fix: use GOPATH for invoking cyclonedx-gomod

* feat: add comment on how to obtain skipped projects in Kafka build

---------

Co-authored-by: Siegfried Weber <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants